Trying to scope the PR to MySQL only so intentionally using a separate method here that gets explicitly called by mysql connector.

AddPartitionsWithRange is used by Postgres as well so enabling it for PG (e.g. when a custom string column is selected as the watermark column) requires more code changes and e2e testing.

Would it make sense to move this method to an utils package within the MySQL connector subtree to prevent accidental usage? Or at least add a comment? I can imagine a happy agent or engineer thinking oh, this is convenient

Sure, can move it to mysql for now and bring it back when we are ready to make it generic.

Would it make sense to move this method to an utils package within the MySQL connector subtree to prevent accidental usage? Or at least add a comment? I can imagine a happy agent or engineer thinking oh, this is convenient

I think it would make sense to add cases for the non-happy path cases just to confirm nothing crash. e.g:

lower:"not an uuid" upper: "018f6e7a-1b2c-7def-8a3b-1c2d3e4f5a6b"
lower: "f47ac10b-58cc-4372-a567-0e02b2c3d479" upper:"018f6e7a-1b2c-7def-8a3b-1c2d3e4f5a6b"
...

oops, had a separate test for this, but looks like i accidentally deleted them when addressing a merge conflict.

added non-happy path testing for buildUuidStringPartitions:

func TestBuildUuidStringPartitionsInvalid(t *testing.T) {
	cases := []struct {
		name string
		minV string
		maxV string
	}{
		{"arbitrary string", "apple", "banana"},
		{"inverted", "f47ac10b-58cc-4372-a567-0e02b2c3d479", "018f6e7a-1b2c-7def-8a3b-1c2d3e4f5a6b"},
		{"too short", "018f6e7a-1b2c", "f47ac10b-58cc"},
		{"too long", "018f6e7a-1b2c-7def-8a3b-1c2d3e4f5a6bb", "f47ac10b-58cc-4372-a567-0e02b2c3d4799"},
	}
	const numPartitions = 32
	for _, tc := range cases {
		t.Run(tc.name, func(t *testing.T) {
			_, err := buildUuidStringPartitions(tc.minV, tc.maxV, hexCasingLower, numPartitions)
			require.Error(t, err)
		})
	}
}

Failure here is intentional since it indicates a code error on our end if buildUuidStringPartitions ends up processing invalid uuids. They should be caught by detectUuidWithHexCasing, which also test for invalid cases in TestDetectUUIDCase. There's also an e2e test Test_MySQL_String_Partition_Key_Arbitrary_FullTable to make sure that we default to full table partition. So we should be good on coverage now.

Is it not possible to normalize the string values both at this partition computation level and at query level so at least for parallel snapshotting we are case insensitive?

We can maintain correctness as long as the min/max values (in the first and last partitions) use the original casing (which is the case today). However, if table collation is case-sensitive, it can lead to data skew.

Skew example: say we have min/max uuids A0000..... and e0000...; let's say we normalize the computed boundaries to lower case and got the following bounds:

• P1 [A0000…, accccc…)
• P2 [accccc…, b99999…)
• P3 [b99999…, c66666…)
• P4 [c66666…, d33333…)
• P5 [d33333…, e0000…]

This means P1 would cover all the upper case uuids, leading to potential skew if table was recently switched to lower case.

Another skew example where we have mixed casing within an UUID: say we have min/max uuid A000a... and E000e...; and again we normalize the computed boundaries to lower case and got the following bounds

• P1 [A000a…, accccc…)
• P2 [accccc…, b99999…)
• P3 [b99999…, c66666…)
• P4 [c66666…, d33333…)
• P5 [d33333…, E000e…]

In this case all data ends up in the first partition, and the other computed partition boundaries return empty results.

We could get a bit smarter with what default casing to use, but it just adds more code complexity. Hence I was thinking we can treat mix-case as arbitrary string for more even partitioning.

Oh wait, perhaps by normalize at computation/query level you probably mean something like where lower(<uuid_col>) >= <start_value_lower_case> and lower(<uuid_col>) < <end_value_lower_case>. This would work but cannot be done cheaply since it can't take advantage of the index, so we end up with full table scans.

Oh wait, perhaps by normalize at computation/query level you probably mean something like where lower(<uuid_col>) >= <start_value_lower_case> and lower(<uuid_col>) < <end_value_lower_case>.

Yes, that's what I meant actually.

since it can't take advantage of the index, so we end up with full table scans.

Great point! I hadn't considered the index!